Processing clinical notes using recurrent neural networks

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting future patient health using neural networks. One of the methods includes receiving electronic health record data for a patient; generating a respective observation embedding for each of the observations, comprising, for each clinical note: processing the sequence of tokens in the clinical note using a clinical note embedding LSTM to generate a respective token embedding for each of the tokens; and generating the observation embedding for the clinical note from the token embeddings; generating an embedded representation, comprising, for each time window: combining the observation embeddings of observations occurring during the time window to generate a patient record embedding; and processing the embedded representation of the electronic health record data using a prediction recurrent neural network to generate a neural network output that characterizes a future health status of the patient.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Application No.62/778,833, filed on Dec. 12, 2018. The disclosure of the priorapplication is considered part of and is incorporated by reference inthe disclosure of this application.

BACKGROUND

This specification relates to processing inputs using a neural network.

Neural networks are machine learning models that employ one or morelayers of nonlinear units to predict an output for a received input.Some neural networks include one or more hidden layers in addition to anoutput layer. The output of each hidden layer is used as input to thenext layer in the network, i.e., the next hidden layer or the outputlayer. Each layer of the network generates an output from a receivedinput in accordance with current values of a respective set ofparameters.

SUMMARY

This specification describes a system of one or more computers in one ormore physical locations that makes predictions that characterize thepredicted future health of a patient based on electronic health recorddata for the patient. In particular, the electronic health data includesclinical notes and, as part of generating the prediction, the systemgenerates an embedding of each clinical note using a clinical noteembedding long short-term memory (LSTM) neural network.

Particular embodiments of the subject matter described in thisspecification can therefore be implemented so as to realize one or moreof the following advantages.

Clinical notes provide immense value to clinicians as a summary of apatient's clinical state and are essential to a patient's care. However,while other parts of a patient's record are uniformly coded and easilyaccessible to predictive models and computer systems (such as billingsystems), clinical notes remain inscrutable to automated analysis.

Clinicians can spend significant amounts of time writing clinical notesthat contain patient histories, assessment of clinical conditions, and adiscussion of recommended therapies. The notes are a critical source ofinformation that is not recorded elsewhere, such as the exactcharacteristics of a joint exam or diagnoses made from clinicalassessment (e.g., alcohol withdrawal). However, these notes are also animperfect source of information, containing redundancies or importingout-of-date data from the structured portion of the electronic healthrecord (EHR). Automated information extraction from the raw, free textof the notes is difficult due to variance in clinicians' styles ofwriting. Clinicians may use their own abbreviations, and may organizethe note in various different ways. Notes are often not written instandard English, as busy clinicians may use non-standard andinconsistent abbreviations (for example, pt and ptnt for patient) orskip over non-essential words such as adverbs.

For the above reasons, conventional techniques that analyze electronichealth record data to make predictions about patient's health have notbeen able to effectively incorporate clinical notes. In particular,conventional techniques for incorporating clinical notes into predictivemodels have not proven to be effective in improving the accuracy of thepredictions generated by these models. In many conventional predictivemodels, clinical notes have been ignored outright; those that use notesmay simply split them into word-level tokens and use the top N mostfrequent tokens (or an expert-selected subset of tokens) as independentpredictor variables in a model, ignoring many subtleties of languagesuch as conjunctions and negations.

The described techniques, on the other hand, effectively incorporateclinical notes to generate highly-accurate predictions about thepatient's future health. In particular, the described systems andtechniques scalably and automatically extract relevant information fromnotes in the context of the entire clinical record by applying ahierarchical recurrent neural network that is able to readvariable-length notes sequentially in tandem with other data from themedical record, without assuming any particular layout, medicalvocabulary, writing style or language rules in the notes. In particular,a low-level LSTM (or other recurrent neural network) generatesembeddings of notes that consider the context of the text in the note,while a higher-level LSTM (or other recurrent neural network) processesthe embeddings of the notes along with embeddings of other types ofobservations to make accurate predictions.

The described approach to modelling clinical notes significantlyimproves predictive performance over strong commonly-used baselines (amodel without clinical notes and one using bags of word embeddings) onmultiple tasks, even those requiring extraction of information that areonly recorded in notes.

The details of one or more embodiments of the subject matter describedin this specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example future health prediction system.

FIG. 2 shows an example of generating a clinical note embedding.

FIG. 3 is a flow diagram of an example process for generating a futurehealth prediction.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1 shows an example future health prediction system 100. The futurehealth prediction system 100 is an example of a system implemented ascomputer programs on one or more computers in one or more locations inwhich the systems, components, and techniques described below areimplemented.

The system 100 makes predictions that characterize the predicted futurehealth of a patient. The predictions are made based on electronic healthrecord data 102 for the patient.

The electronic health record data includes one or more observations ofeach of a plurality of different feature types, e.g., medications,procedures, and so on. In the example of FIG. 1, the electronic healthrecord data includes features of the medication feature type, theprocedures feature type, the clinical notes feature type, and,optionally, features gathered from other clinical data modalities.

More specifically, the plurality of different feature types includes aclinical note feature type and each observation of the clinical notefeature type is a clinical note that includes a respective sequence oftext tokens. Generally, clinical notes are text sequences generated by aclinician, i.e., a healthcare professional, and often includeinformation that is relevant to the patient's current and future health.Unlike the other feature types, the clinical notes are generally raw,free text. As one example, the clinician may directly generate the textsequence by inputting text through an input modality, e.g., a keyboardor touchscreen, on a user computer. As another example, the clinicianmay speak the contents of the note into a microphone of the usercomputer and the system 100 or another system may generate the textsequence by applying automatic speech recognition to the speech of theclinician.

To generate the prediction, the system generates a respectiveobservation embedding 104 for each of the observations and thengenerates an embedded representation of the electronic health recorddata using the observation embeddings 104. In particular, the embeddedrepresentation includes a respective patient record embedding 110corresponding to each of a plurality of time windows and, to generatethe patient record embedding 110 for a given time window, the systemcombines the observation embeddings of observations occurring during thetime window. The set of observations occurring during a given timewindow is referred to in FIG. 1 as an “hourly bag,” but the time windowcan have any appropriate length, e.g., one hour, four hours, six hours,twelve hours, or twenty-four hours. In some cases, any observationsoccurring more than a threshold amount of time, e.g., one thousand hoursor more, before the current time are grouped into a single, “history”time window that precedes all of the other time windows in the embeddedrepresentation.

An “embedding” as used in this specification is a numeric representationin a particular space, i.e., an ordered collection of numeric values,e.g., a vector of floating point values or other type of numeric value,having a particular dimensionality.

In particular, when there are multiple observations of the same featuretype occurring within a given time window, the system 100 combines theobservation embeddings of that feature type, e.g., by averaging orsumming the embeddings, to generate a combined observation. When only asingle observation of the feature type occurs during the time window,the system 100 uses the single observation embedding of that featuretype as the combined observation embedding. When no observations of agiven type occur during a time window, the system 100 can include adefault embedding as the combined embedding for that feature type.

The system 100 then combines, e.g., concatenates, the combined featureembeddings for the feature types to generate the patient recordembedding 110 for the given time window.

The system 100 then processes the embedded representation of theelectronic health record data using a prediction recurrent neuralnetwork 120 to generate a neural network output 150 that characterizesthe future health status of the patient after the last time window inthe embedded representation.

The prediction recurrent neural network 120 can have any recurrentneural network architecture that allows the prediction recurrent neuralnetwork 120 to map a sequence of patient record embeddings to the neuralnetwork output 150. For example, the neural network 120 can be a longshort-term memory (LSTM) neural network or another type of recurrentneural network, e.g., a vanilla recurrent neural network, a gatedrecurrent unit neural network, and so on, with an output layer that hasthe appropriate number of neurons.

The future health status of the patient is the status of the patient'shealth with respect to one or more predetermined aspects.

As one example, the network output 150 can predict a likelihood ofinpatient mortality. In other words, network output 150 can include ascore that represents the likelihood of a mortality event while thepatient is admitted at a medical facility.

As another example, the network output 150 can predict a dischargediagnosis at the time the patient is discharged from care, e.g., bygenerating a probability distribution over a set of possible diagnoses.In other words, the network output 150 can include a probabilitydistribution over a set of possible diagnoses, with the probability foreach diagnosis being the probability that the diagnosis will be adiagnosis for the patient at the time that the patient is dischargedfrom care.

As another example, the network output 150 can predict a likelihood thata particular adverse health event occurs to the patient, e.g., an organinjury, cardiac arrest, a stroke, or mortality, within some specifiedtime from the last time window in the health record data for thepatient. In other words, the network output 150 can include a score foreach of one or more adverse health events that represents a likelihoodthat the adverse health event occurs to the patient within somespecified time from the last time window.

Because the clinical notes are text sequences of variable length,conventional systems have not been able to effectively incorporate theinformation in clinical notes into a framework such as the one describedabove.

In order to account for the variable, free form nature of clinicalnotes, to generate the observation embedding of a clinical note, thesystem processes the sequence of text tokens in the clinical note usinga clinical note embedding long short-term memory (LSTM) neural networkto generate a respective token embedding for each of the text tokens andgenerates the observation embedding for the clinical note from the tokenembeddings for the text tokens in the clinical note, e.g., byaggregating the token embeddings using a learned attention weighting.

This will be described in more detail below with reference to FIG. 2.

By generating the observation embeddings of the clinical note asdescribed in this specification, the system 100 can effectively andaccurately predict the risk the future health status of the patient.Accordingly, by employing the described techniques, clinicians can beprovided with accurate prediction data that can then allow them toeffectively treat the patient, e.g., by taking preventative action inadvance of the future health actually occurring.

For observations that are of a type other than clinical notes, thesystem 100 can generate the observation embedding in a conventionalmanner. For example, to generate the observation embedding for anyobservations that are discrete, i.e., that can only take one or morevalues from a discrete set of possible values, the system 100 maps theobservation to a learned embedding for the observation. As anotherexample, to generate an observation embedding for any observations thatare continuous, i.e., that are medical test results or other medicaldata that can take any value within some range of values, the system 100standardizes the observation using a cohort mean and standard deviationfor the feature type to generate the observation embedding. In otherwords, the system generates a standardized value by normalizing thecontinuous observation using the cohort mean and standard deviation forthe feature type and then uses the standardized value as the observationembedding.

Once the neural network output 150 has been generated, the system 100can provide the information in the network output 150 for use by amedical professional in treating the patient or store the information inthe network output 150 for later access by a medical professional. Asone example, the system 100 can determine whether any of the scores orprobabilities in the output 150 exceed a corresponding threshold and, ifso, transmit an alert for presentation to a user, e.g., to a usercomputer of a physician or other medical personnel.

As another example, the system 100 can generate a user interfacepresentation based on the data in the neural network output 150, e.g., apresentation that conveys the patient's predicted future health, andthen provide the user interface presentation for display on the usercomputer.

In some implementations, the system 100 continually updates the neuralnetwork output 150 as new electronic health record data for the patientbecomes available. For example, the system 100 can generate an initialembedded representation and generate an initial neural network outputwhen a patient is admitted for treatment or at another initial timepoint. The system 100 can then obtain new data at the expiration of eachsubsequent time window and generate updated neural network outputs foreach of the subsequent time windows until the patient is discharged oruntil some other termination criteria are satisfied.

FIG. 2 shows an example of generating an observation embedding 104 for aclinical note 202.

As described above, clinical notes are text sequences generated by aclinician, i.e., a healthcare professional, and often includeinformation that is relevant to the patient's current and future health.A text sequence is a sequence of text tokens, i.e., words or characters.Unlike the other feature types, the clinical notes are generally raw,free text. In the particular example of FIG. 2, the clinical note 202 isa text sequence that includes the fragment “patient presents withleukocytosis.”

To generate the observation embedding 104 for the clinical note 202, thesystem processes the sequence of text tokens in the clinical note 202using a clinical note embedding long short-term memory (LSTM) neuralnetwork 220 to generate a respective token embedding for each of thetext tokens.

As part of the processing of the observation embedding 104, the neuralnetwork 220 generates an initial embedding 210 for each of the texttokens, i.e., by mapping each token to an embedding for the token usinga look-up table or an embedding neural network layer. In the example ofFIG. 2, the neural network 220 has generated respective initialembeddings 210 for each of the tokens “patient,” “presents,” “with,” and“leukocytosis.” These embeddings can either be learned jointly with thetraining of the clinical note LSTM neural network 220 or can be fixedprior to the training of the clinical note LSTM neural network 220.

In some implementations, the clinical note LSTM neural network 220 is auni-directional LSTM neural network, i.e., an LSTM neural network thatprocesses the text tokens in the order in which the tokens occur in theclinical note. In some other implementations, the clinical noteembedding LSTM neural network is a bi-directional LSTM neural networkthat processes the text tokens both in a forward order, i.e., in theorder in which the tokens occur in the clinical note, and in a backwardorder, i.e., in the opposite order from the order in which the tokensoccur, to generate the respective token embeddings.

While the neural network 220 is described as being an LSTM, the neuralnetwork 220 can generally be any type of recurrent neural network, e.g.,a vanilla recurrent neural network or a gated recurrent unit neuralnetwork.

Because the neural network 220 is a recurrent neural network andprocesses the tokens in order, the token embeddings generated by theneural network 220 are dependent on the order in which the tokens occurin the note and therefore provide a more informational representationthan conventional, e.g., bag-of-words or computing statistics just fromthe top N most frequently occurring tokens, techniques.

The system then generates the observation embedding 104 for the clinicalnote 202 from the token embeddings for the text tokens in the clinicalnote.

In some implementations, the system combines the token embeddings in afixed manner to generate the observation embedding 104, e.g., computesan average or other measure of central tendency of the token embeddings.

In some other implementations, like in the example of FIG. 2, the systemapplies a learned attention weighting 230 to the token embeddings togenerate the observation embedding 104. In other words, the systemaggregates the token embeddings for the text tokens in the clinical noteusing a learned attention weighting to generate the observationembedding 104. Generally, the learned attention weighting generates arespective weight for each token embedding that is based on one or morelearned values, i.e., learned during the training of the neural network220, and the token embeddings for the tokens in the note.

More specifically, to apply the learned attention weighting 230, thesystem generates a respective initial weight for each token embedding bycomputing a dot product between a learned context vector and the tokenembedding.

The system then generates a respective normalized weight for each tokenembedding using the initial weight for the token embedding, the learnedcontext vector, a learned bias vector, and the initial weights for theother token embeddings. In particular, the normalized weight at for thetoken embedding ht can satisfy:

${a_{t} = \frac{\exp\left( {q^{T}h_{t}} \right)}{{\exp\left( {q^{T}b} \right)} + {\sum\limits_{t^{\prime} = 1}^{T}{\exp\left( {q^{T}h_{t^{\prime}}} \right)}}}},$where q is the learned context vector, b is the learned bias vector, andT is the total number of tokens in the clinical note.

The system then computes a weighted sum of the token embeddings, witheach token embedding being weighted by the corresponding normalizedweight. This weighted sum can then be used as the observation embedding104.

In order for the system to use the clinical note LSTM neural network andthe prediction recurrent neural network to make accurate predictions,the system trains these two neural networks jointly on training datathat includes ground truth network outputs. That is, the training dataincludes a set of electronic health record data, and for each electronichealth record data in the set, a ground truth network output that shouldbe generated by the prediction recurrent neural network for theelectronic health record data. The system can train the neural networksjointly on this training data through supervised learning by minimizingan appropriate loss function that measures errors between the networkoutputs generated by the prediction neural network and the correspondingground truth outputs, e.g., a cross-entropy loss function.

When the attention weighting 230 is used, the system can train theclinical note LSTM neural network, the prediction recurrent neuralnetwork, and the attention weighting jointly on the same training datathat includes ground truth network outputs.

In some implementations, prior to this joint training, the systempre-trains the clinical note LSTM neural network using unsupervisedlearning on a set of training sequences extracted from training clinicalnotes to predict, for each training sequence, at least a next word thatfollows the last word in the training sequence in the training clinicalnote. When the clinical note LSTM is a bi-directional LSTM, the systemcan also train the network to predict the word that precedes the firstword in the training sequence in the clinical note. This pre-trainingcan enrich the representations generated by the clinical note LSTM andcan allow the embeddings generated by this LSTM to encode a flexible,generalizable language representation of the corresponding clinicalnotes.

FIG. 3 is a flow diagram of an example process 300 for generating afuture health prediction. For convenience, the process 300 will bedescribed as being performed by a system of one or more computerslocated in one or more locations. For example, a future healthprediction system, e.g., the future health prediction system 100 of FIG.1, appropriately programmed, can perform the process 300.

The system receives electronic health record data for a patient (step302). The electronic health record data includes one or moreobservations of each of a plurality of different feature types and theplurality of different feature types includes a clinical note featuretype. Each observation of the clinical note feature type is a clinicalnote that includes a respective sequence of text tokens.

The system generates a respective observation embedding for each of theobservations (step 304). To generate the observation embedding for anyobservations that are clinical notes, the system processes the sequenceof text tokens in the clinical note using a clinical note embedding longshort-term memory (LSTM) neural network to generate a respective tokenembedding for each of the text tokens and generates the observationembedding for the clinical note from the token embeddings for the texttokens in the clinical note. To generate an observation embedding forany observations that are discrete, i.e., that can only take one or morevalues from a discrete set of possible values, the system maps theobservation to a learned embedding for the observation. To generate anobservation embedding for any observations that are continuous, i.e.,that are diagnostic test results or other medical data that can take anyvalue within some range of values, the system standardizes theobservation using a cohort mean and standard deviation for the featuretype.

The system generates an embedded representation of the electronic healthrecord data (step 306). The embedded representation includes arespective patient record embedding corresponding to each of multipletime windows, e.g., time windows that each cover the same, fixed amountof time. Within the embedded representation, the patient recordembeddings can be arranged in a sequence starting from the earliest timewindow.

To generate a patient record embedding for any given time window, thesystem combines the observation embeddings of observations occurringduring the time window. In particular, for each feature type, the systemcan combine all of the observation embeddings of observations of thefeature type that occurred during the time window to generate a combinedobservation embedding for the feature type and combine the combinedobservation embeddings to generate the patient record embedding. Forexample, to combine all observation embeddings of observations of thefeature type that occurred during the time window, the system canaverage all observation embeddings of observations of the feature typethat occurred during the time window. As another example, to combine thecombined observation embeddings to generate the patient record embeddingthe system can concatenate the combined observation embeddings.

The system processes the embedded representation of the electronichealth record data using a prediction recurrent neural network togenerate a neural network output (step 308). Generally, the neuralnetwork output characterizes a future health status of the patient afterthe last time window in the embedded representation. The future healthstatus of the patient is the status of the patient's health with respectto one or more predetermined aspects. For example, the network outputcan predict one or more of: a likelihood of inpatient mortality, adischarge diagnosis at the time the patient is discharged from care, alikelihood that a particular adverse health event occurs, e.g., an organinjury, cardiac arrest, a stroke, or mortality within some specifiedtime of the last time window, and so on.

This specification uses the term “configured” in connection with systemsand computer program components. For a system of one or more computersto be configured to perform particular operations or actions means thatthe system has installed on it software, firmware, hardware, or acombination of them that in operation cause the system to perform theoperations or actions. For one or more computer programs to beconfigured to perform particular operations or actions means that theone or more programs include instructions that, when executed by dataprocessing apparatus, cause the apparatus to perform the operations oractions.

Embodiments of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly-embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Embodiments of the subject matter described in thisspecification can be implemented as one or more computer programs. Theone or more computer programs can comprise one or more modules ofcomputer program instructions encoded on a tangible non transitorystorage medium for execution by, or to control the operation of, dataprocessing apparatus. The computer storage medium can be amachine-readable storage device, a machine-readable storage substrate, arandom or serial access memory device, or a combination of one or moreof them. Alternatively or in addition, the program instructions can beencoded on an artificially generated propagated signal, e.g., amachine-generated electrical, optical, or electromagnetic signal, thatis generated to encode information for transmission to suitable receiverapparatus for execution by a data processing apparatus.

The term “data processing apparatus” refers to data processing hardwareand encompasses all kinds of apparatus, devices, and machines forprocessing data, including by way of example a programmable processor, acomputer, or multiple processors or computers. The apparatus can alsobe, or further include, special purpose logic circuitry, e.g., an FPGA(field programmable gate array) or an ASIC (application specificintegrated circuit). The apparatus can optionally include, in additionto hardware, code that creates an execution environment for computerprograms, e.g., code that constitutes processor firmware, a protocolstack, a database management system, an operating system, or acombination of one or more of them.

A computer program, which may also be referred to or described as aprogram, software, a software application, an app, a module, a softwaremodule, a script, or code, can be written in any form of programminglanguage, including compiled or interpreted languages, or declarative orprocedural languages; and it can be deployed in any form, including as astand alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A program may, but neednot, correspond to a file in a file system. A program can be stored in aportion of a file that holds other programs or data, e.g., one or morescripts stored in a markup language document, in a single file dedicatedto the program in question, or in multiple coordinated files, e.g.,files that store one or more modules, sub programs, or portions of code.A computer program can be deployed to be executed on one computer or onmultiple computers that are located at one site or distributed acrossmultiple sites and interconnected by a data communication network.

In this specification, the term “database” is used broadly to refer toany collection of data: the data does not need to be structured in anyparticular way, or structured at all, and it can be stored on storagedevices in one or more locations. Thus, for example, the index databasecan include multiple collections of data, each of which may be organizedand accessed differently.

Similarly, in this specification the term “engine” is used broadly torefer to a software-based system, subsystem, or process that isprogrammed to perform one or more specific functions. Generally, anengine will be implemented as one or more software modules orcomponents, installed on one or more computers in one or more locations.In some cases, one or more computers will be dedicated to a particularengine; in other cases, multiple engines can be installed and running onthe same computer or computers.

The processes and logic flows described in this specification can beperformed by one or more programmable computers executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby special purpose logic circuitry, e.g., an FPGA or an ASIC, or by acombination of special purpose logic circuitry and one or moreprogrammed computers.

Computers suitable for the execution of a computer program can be basedon general or special purpose microprocessors or both, or any other kindof central processing unit. Generally, a central processing unit willreceive instructions and data from a read only memory or a random accessmemory or both. The essential elements of a computer are a centralprocessing unit for performing or executing instructions and one or morememory devices for storing instructions and data. The central processingunit and the memory can be supplemented by, or incorporated in, specialpurpose logic circuitry. Generally, a computer will also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto optical disks, or optical disks. However, a computer need nothave such devices. Moreover, a computer can be embedded in anotherdevice, e.g., a mobile telephone, a personal digital assistant (PDA), amobile audio or video player, a game console, a Global PositioningSystem (GPS) receiver, or a portable storage device, e.g., a universalserial bus (USB) flash drive, to name just a few.

Computer readable media suitable for storing computer programinstructions and data include all forms of non volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto optical disks; andCD ROM and DVD-ROM disks.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending web pages to a web browser on auser's device in response to requests received from the web browser.Also, a computer can interact with a user by sending text messages orother forms of message to a personal device, e.g., a smartphone that isrunning a messaging application, and receiving responsive messages fromthe user in return.

Data processing apparatus for implementing machine learning models canalso include, for example, special-purpose hardware accelerator unitsfor processing common and compute-intensive parts of machine learningtraining or production, i.e., inference, workloads.

Machine learning models can be implemented and deployed using a machinelearning framework, e.g., a TensorFlow framework, a Microsoft CognitiveToolkit framework, an Apache Singa framework, or an Apache MXNetframework.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front end component, e.g., aclient computer having a graphical user interface, a web browser, or anapp through which a user can interact with an implementation of thesubject matter described in this specification, or any combination ofone or more such back end, middleware, or front end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (LAN) and a widearea network (WAN), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server transmits data, e.g., an HTML page, to a userdevice, e.g., for purposes of displaying data to and receiving userinput from a user interacting with the device, which acts as a client.Data generated at the user device, e.g., a result of the userinteraction, can be received at the server from the device.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinvention or on the scope of what may be claimed, but rather asdescriptions of features that may be specific to particular embodimentsof particular inventions. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially be claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings and recited inthe claims in a particular order, this should not be understood asrequiring that such operations be performed in the particular ordershown or in sequential order, or that all illustrated operations beperformed, to achieve desirable results. In certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system modules and components in the embodimentsdescribed above should not be understood as requiring such separation inall embodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In some cases, multitasking and parallel processing may beadvantageous.

What is claimed is:
 1. A system comprising one or more computers and oneor more storage devices storing instructions that, when executed by theone or more computers, cause the one or more computers to performoperations comprising: receiving electronic health record data for apatient, the electronic health record data comprising one or moreobservations of each of a plurality of different feature types, and theplurality of different feature types including a clinical note featuretype, wherein each observation of the clinical note feature type is aclinical note comprising a respective sequence of text tokens;generating a respective observation embedding for each of theobservations, comprising, for each clinical note: processing thesequence of text tokens in the clinical note using a clinical noteembedding long short-term memory (LSTM) neural network to generate arespective token embedding for each of the text tokens; and generatingthe observation embedding for the clinical note from the tokenembeddings for the text tokens in the clinical note, comprisingcomputing a weighted sum of the token embeddings, with each tokenembedding being weighted by a corresponding weight; generating anembedded representation of the electronic health record data, whereinthe embedded representation comprises a respective patient recordembedding corresponding to each of a plurality of time windows, andwherein generating the embedded representation comprises, for each timewindow: combining the observation embeddings of observations occurringduring the time window to generate the patient record embeddingcorresponding to the time window; and processing the embeddedrepresentation of the electronic health record data using a predictionrecurrent neural network to generate a neural network output thatcharacterizes a future health status of the patient after the last timewindow in the embedded representation.
 2. The system of claim 1, whereincombining the observation embeddings to generate the patient recordembedding corresponding to the time window comprises: for each featuretype, combining all observation embeddings of observations of thefeature type that occurred during the time window to generate a combinedobservation embedding for the feature type; and combining the combinedobservation embeddings to generate the patient record embedding.
 3. Thesystem of claim 2, wherein combining all observation embeddings ofobservations of the feature type that occurred during the time window togenerate a combined observation embedding for the feature type comprisesaveraging all observation embeddings of observations of the feature typethat occurred during the time window.
 4. The system of claim 2, whereincombining the combined observation embeddings to generate the patientrecord embedding comprises concatenating the combined observationembeddings.
 5. The system of claim 1, wherein the clinical noteembedding LSTM neural network is a bi-directional LSTM neural network.6. The system of claim 1, wherein generating the observation embeddingfor the clinical note from the token embeddings for the text tokens inthe clinical note comprises: aggregating the token embeddings for thetext tokens in the clinical note using a learned attention weighting togenerate the observation embedding.
 7. The system of claim 6, whereinaggregating using the learned attention weighting comprises: generatinga respective initial weight for each token embedding by computing a dotproduct between a learned context vector and the token embedding;generating a respective normalized weight for each token embedding usingthe initial weight for the token embedding, the learned context vector,a learned bias vector, and the initial weights for the other tokenembeddings; and computing the weighted sum of the token embeddings, witheach token embedding being weighted by the corresponding normalizedweight.
 8. The system of claim 6, the operations further comprising:training the clinical note LSTM neural network and the predictionrecurrent neural network jointly on training data including ground truthnetwork outputs.
 9. The system of claim 8, wherein training the clinicalnote LSTM neural network and the prediction recurrent neural networkjointly on training data including ground truth network outputscomprises: training the clinical note LSTM neural network, theprediction recurrent neural network, and the attention weighting jointlyon the training data including ground truth network outputs.
 10. Thesystem of claim 8, the operations further comprising: prior to thetraining, pre-training the clinical note LSTM neural network usingunsupervised learning on a set of training sequences extracted fromtraining clinical notes to predict, for each training sequence, at leasta next word that follows the last word in the training sequence in thetraining clinical note.
 11. The system of claim 1, wherein the networkoutput comprises a prediction of inpatient mortality of the patient. 12.The system of claim 1, wherein the network output comprises a predictionof a primary diagnosis of the patient at discharge.
 13. The system ofclaim 1, wherein the network output comprises a prediction of a set ofdiagnoses for the patient at discharge.
 14. The system of claim 1,wherein generating a respective observation embedding for each of theobservations, comprises, for each observation that is of a feature typethat is discrete: mapping the observation to a learned embedding for theobservation.
 15. The system of claim 1, wherein generating a respectiveobservation embedding for each of the observations, comprises, for eachobservation that of a feature type that is continuous: standardizing theobservation using a cohort mean and standard deviation for the featuretype.
 16. One or more non-transitory computer-readable storage mediastoring instructions that when executed by one or more computers causethe one or more computers to perform operations comprising: receivingelectronic health record data for a patient, the electronic healthrecord data comprising one or more observations of each of a pluralityof different feature types, and the plurality of different feature typesincluding a clinical note feature type, wherein each observation of theclinical note feature type is a clinical note comprising a respectivesequence of text tokens; generating a respective observation embeddingfor each of the observations, comprising, for each clinical note:processing the sequence of text tokens in the clinical note using aclinical note embedding long short-term memory (LSTM) neural network togenerate a respective token embedding for each of the text tokens; andgenerating the observation embedding for the clinical note from thetoken embeddings for the text tokens in the clinical note, comprisingcomputing a weighted sum of the token embeddings, with each tokenembedding being weighted by a corresponding weight; generating anembedded representation of the electronic health record data, whereinthe embedded representation comprises a respective patient recordembedding corresponding to each of a plurality of time windows, andwherein generating the embedded representation comprises, for each timewindow: combining the observation embeddings of observations occurringduring the time window to generate the patient record embeddingcorresponding to the time window; and processing the embeddedrepresentation of the electronic health record data using a predictionrecurrent neural network to generate a neural network output thatcharacterizes a future health status of the patient after the last timewindow in the embedded representation.
 17. A computer-implemented methodcomprising: receiving electronic health record data for a patient,  theelectronic health record data comprising one or more observations ofeach of a plurality of different feature types, and  the plurality ofdifferent feature types including a clinical note feature type, whereineach observation of the clinical note feature type is a clinical notecomprising a respective sequence of text tokens; generating a respectiveobservation embedding for each of the observations, comprising, for eachclinical note:  processing the sequence of text tokens in the clinicalnote using a clinical note embedding long short-term memory (LSTM)neural network to generate a respective token embedding for each of thetext tokens; and  generating the observation embedding for the clinicalnote from the token embeddings for the text tokens in the clinical note,comprising computing a weighted sum of the token embeddings, with eachtoken embedding being weighted by a corresponding weight; generating anembedded representation of the electronic health record data, whereinthe embedded representation comprises a respective patient recordembedding corresponding to each of a plurality of time windows, andwherein generating the embedded representation comprises, for each timewindow:  combining the observation embeddings of observations occurringduring the time window to generate the patient record embeddingcorresponding to the time window; and processing the embeddedrepresentation of the electronic health record data using a predictionrecurrent neural network to generate a neural network output thatcharacterizes a future health status of the patient after the last timewindow in the embedded representation.
 18. The method of claim 17,wherein combining the observation embeddings to generate the patientrecord embedding corresponding to the time window comprises: for eachfeature type, combining all observation embeddings of observations ofthe feature type that occurred during the time window to generate acombined observation embedding for the feature type; and combining thecombined observation embeddings to generate the patient recordembedding.
 19. The method of claim 18, wherein combining all observationembeddings of observations of the feature type that occurred during thetime window to generate a combined observation embedding for the featuretype comprises averaging all observation embeddings of observations ofthe feature type that occurred during the time window.
 20. The method ofclaim 17, wherein the clinical note embedding LSTM neural network is abi-directional LSTM neural network.